Harden LSUNClass key cache: store under dataset root and load with weights_only - #9628
Harden LSUNClass key cache: store under dataset root and load with weights_only#9628fjankovi wants to merge 1 commit into
Conversation
…ights_only
`LSUNClass.__init__` derived its key-cache filename from the ASCII letters of
`root` and read/wrote it in the current working directory via `pickle.load`
/ `pickle.dump`:
cache_file = "_cache_" + "".join(c for c in root if c in string.ascii_letters)
if os.path.isfile(cache_file):
self.keys = pickle.load(open(cache_file, "rb"))
Two problems:
- The path is resolved against the process CWD, not `root`, and the name is a
predictable, lossy transform of `root`. Anyone able to write a file into the
directory the user runs from (a shared scratch dir, a cloned repo, a CI
workspace) can pre-plant `_cache_<letters>` and have it deserialized on the
next `LSUN()` call. `pickle.load` on an attacker-controlled file is arbitrary
code execution (CWE-502); there is no integrity check on the path.
- Even setting that aside, the cache is a plain pickle with no restriction.
Fix, matching the pattern already used by ImageNet/MNIST/PhotoTour in this
package:
- Store the cache next to the LMDB store as `root/_cache_keys.pt` (a trusted,
per-class location; no CWD, no cross-root name collisions).
- Load with `torch.load(..., weights_only=True)` so a cache file can never
execute code on load.
- Degrade gracefully when `root` is read-only (skip caching, re-enumerate).
The cache only memoizes keys that are fully re-derivable from the trusted LMDB
store, so old CWD caches are simply not found under the new path and are
regenerated safely on first use — no migration needed. `torch.load` round-trips
the raw `bytes` LMDB keys, including non-UTF-8 keys.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/9628
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Hi @fjankovi! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
Summary
LSUNClasscaches the enumerated LMDB keys in a pickle file whose path is derived fromrootbut resolved against the current working directory, and reads it back withpickle.load:This has two issues:
root, and the name is a lossy transform ofroot(letters only). A file dropped into the directory the user happens to run from — a shared scratch dir, a cloned example repo, a CI workspace — is loaded on the nextLSUN()/LSUNClass()call. Because it's unpickled with no integrity check, that is arbitrary code execution on load (CWE-502). The user never asked to load anything; the cache is written and re-read as a side effect.Fix
This mirrors what the sibling datasets in this package already do (
ImageNet,MNIST,PhotoTourall usetorch.load(..., weights_only=True)):root/_cache_keys.pt— a trusted, per-class location, with no CWD exposure and no cross-rootname collisions.torch.load(..., weights_only=True), so a cache file can never execute code on load.rootis read-only.The cache only memoizes keys that are fully re-derivable from the trusted LMDB store, so pre-existing CWD caches are simply not found under the new path and are regenerated safely on first use — no migration required.
torch.loadround-trips the rawbytesLMDB keys, including non-UTF-8 keys.Test plan
_cache_keys.ptis written underroot, and a second construction reads it back with the non-UTF-8 key intact._cache_<letters>file in the CWD is now ignored (the loader looks underroot), so a poisoned CWD cache is no longer read.